Use of Machine Learning for Classification of Magneto Cardiograms

ABSTRACT

The use of machine learning for pattern recognition in magnetocardiography (MCG) that measures magnetic fields emitted by the electrophysiological activity of the heart is disclosed herein. Direct kernel methods are used to separate abnormal MCG heart patterns from normal ones. For unsupervised learning, Direct Kernel based Self-Organizing Maps are introduced. For supervised learning Direct Kernel Partial Least Squares and (Direct) Kernel Ridge Regression are used. These results are then compared with classical Support Vector Machines and Kernel Partial Least Squares. The hyper-parameters for these methods are tuned on a validation subset of the training data before testing. Also investigated is the most effective pre-processing, using local, vertical, horizontal and two-dimensional (global) Mahanalobis scaling, wavelet transforms, and variable selection by filtering.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of pending application U.S. Ser. No.14/288,697 filed May 28, 2014, now U.S. Pat. No. 9,173,614 issued Nov.3, 2015. Said U.S. Ser. No. 14/288,697 is a continuation of U.S. Ser.No. 13/772,138 filed Feb. 20, 2013, now U.S. Pat. No. 8,744,557 issuedJun. 3, 2014. Said U.S. Ser. No. 13/772,138 is a continuation of U.S.Ser. No. 12/819,095 filed Jun. 18, 2010, now U.S. Pat. No. 8,391,963issued Mar. 5, 2013. Said U.S. Ser. No. 12/819,095 is a continuation ofU.S. Ser. No. 10/561,285 filed Dec. 20, 2005, now U.S. Pat. No.7,742,806 issued Jun. 22, 2010. Said U.S. Ser. No. 10/561,285 is a USnational stage application of PCT/US04/21,307 filed Jul. 1, 2004, whichin turn claims benefit of U.S. 60/483,961 filed Jul. 1, 2003. Theforegoing are all hereby incorporated by reference.

BACKGROUND OF THE INVENTION

Although magnetocardiography (MCG) was introduced in the early 1960's asa possible diagnostic tool, it took almost thirty years to successfullydemonstrate its clinical value. Today, it represents one of the emergingnew technologies in cardiology employed by physicians in hospitalsaround the world. The clinical application of MCG method hassignificantly benefited from modern multichannel sensor technology,sophisticated software, as well as from recent improvements in hardwareallowing the use of the device without a magnetically-shielded room.

MCG studies are fast, safe and completely noninvasive. Consequently,this offers great convenience for the patient. Currently, many groupswork on establishing libraries of reference data and datastandardization. There are several clinical applications for which MCGhas already provided clinically-useful results. For example, MCG candiagnose and localize acute myocardial infarctions, separate myocardialinfarction patients with and without susceptibility of malignantventricular arrhythmias, detect ventricular hypertrophy and rejectionafter heart transplant, localize the site of ventricular pre-excitationand many types of cardiac arrhythmias, as well as reveal fetalarrhythmias and conduction disturbances [40]. In addition, several otherclinical applications of MCG have recently been studied: detection andrisk stratification of cardiomyopathies (dilated, hypertrophic,arrhythmogenic, diabetic), risk stratification after idiopathicventricular fibrillation, detection and localization of myocardialviability, and follow-up of fetal growth and neural integrity. Somestudies have clearly indicated that MCG is very sensitive to changes inrepolarization, e.g., after myocardial infarction or in a hereditarylong-QT syndrome [42]. The most relevant overview of MCG applicationsand currently-used analysis techniques can be found in [41].

An important challenge, however, is to reduce or eliminate thevariability introduced by human interpretation of MCG data, and tosignificantly improve the machine-based classification performance andquality of generalization, while maintaining computer processing timesthat are compatible with real-time diagnosis.

Three basic steps are always performed when applying artificialintelligence (machine learning) to measured data: 1. measurement of thedata, 2. pre-processing of the measured data, 3. training of theadaptive classifier. Patents incorporating this basic approach toEKG/ECG data or other biological data include U.S. Pat. Nos. 5,092,343;5,280,792; 5,465,308; 5,680,866; 5,819,007; 6,128,608; 6,248,063;6,443,889; 6,572,560; 6,714,925; and 6,728,691.

The use of artificial intelligence for analysis of MCG field patterns isquite limited to date. One reference for the application of artificialintelligence for analysis of biomagnetic signals is U.S. Pat. No.5,417,211, which discloses a method for classifying field patternsgenerated by electrophysiological activities occurring inside the bodyof a living subject including the steps of measuring field patternsarising as a result of the electrophysiological activities outside thebody of the subject using a multi-channel measuring apparatus,generating feature vectors corresponding to the measured field patterns,supplying the feature vectors to an adaptive classifier, and trainingthe adaptive classifier with training field patterns which have beengenerated by a localizable surrogate model of the electrophysiologicalactivity. The method includes the further step of generating aprobability value for each field pattern at an output of the adaptiveclassifier which indicates the probability with which each field patterncan be generated by a selected localizable surrogate model. Like theEKG/ECG references cited above, this discusses the general applicabilityof machine learning to the measured data, but does not present thespecifics of how to improve the classification performance and qualityof generalization.

In all cases, the two key measures which determine success are theclassification performance and quality of generalization. While trainingon non-optimally pre-processed data leads to poor classificationresults, so-called overtraining prevents the adaptive classifier fromgeneralizing to the proper recognition of real-world data.

The key to success lies in an optimal pre-processing of the data, whichhas not yet been achieved to date by any of the references cited herein.It is critically important to identify all features that determine theclass to which the investigated dataset belongs. It is neither obviousnor trivial to identify those features. Moreover, these features mayvary from biological system to biological system, and from one type ofmeasured data to another. In consequence, most artificial intelligencebased procedures differ in how the pre-processing is performed.

As will be disclosed in detail herein, the use of kernel transforms andwavelet transforms to preprocess data for machine learning provides thebasis for a successful machine learning approach which significantlyimproves on the prior art in terms of accurate classification, qualityof generalization, and speed of processing. This has not been disclosedor suggested in any of the prior art cited herein.

SUMMARY OF THE INVENTION

The use of machine learning for pattern recognition inmagnetocardiography (MCG) that measures magnetic fields emitted by theelectrophysiological activity of the heart is disclosed herein. Directkernel methods are used to separate abnormal MCG heart patterns fromnormal ones. For unsupervised learning, Direct Kernel basedSelf-Organizing Maps are introduced. For supervised learning DirectKernel Partial Least Squares and (Direct) Kernel Ridge Regression areused. These results are then compared with classical Support VectorMachines and Kernel Partial Least Squares. The hyper-parameters forthese methods are tuned on a validation subset of the training databefore testing. Also investigated is pre-processing, using local,vertical, horizontal and two-dimensional (global) Mahanalobis scaling,wavelet transforms, and variable selection by filtering. The results,similar for all three methods, were encouraging, exceeding the qualityof classification achieved by the trained experts.

Disclosed herein is a device and associated method for classifyingcardiography data, comprising applying a kernel transform to sensed dataacquired from sensors sensing electromagnetic heart activity, resultingin transformed data, prior to classifying the transformed data usingmachine learning.

BRIEF DESCRIPTION OF THE DRAWINGS

The features of the invention believed to be novel are set forth in theappended claims. The invention, however, together with further objectsand advantages thereof, may best be understood by reference to thefollowing description taken in conjunction with the accompanyingdrawing(s) summarized below.

FIG. 1 illustrates filtered and averaged temporal MCG traces over onecardiac cycle collected in 36 channels comprising a 6×6 grid.

FIG. 2 is a chart illustrating numbers of correct patterns and misses(for negative and positive cases on 36 test data) as well as executiontimes for magnetocardiogram data. Support vector machine library(SVMLib) and kernel partial least square (K-PLS) use time domain and theremaining methods use D-4 wavelet domain.

FIG. 3 is a chart illustrating quality measures for different methodsfor creating predictive models for magneto-cardiogram data.

FIG. 4 is an error plot for 35 test cases, based on K-PLS forwavelet-transformed data.

FIG. 5 is a receiver operator characteristics (ROC) curve showingpossible trade-offs between false positive and false negatives.

FIG. 6 is a Projection of 73 training data, based on (left) DirectKernel Principal Component Analysis (DK-PCA), and (right) Direct KernelPLS (DK-PLS). Diseased cases are shown as filled circles. The test dataare not shown.

FIG. 7 illustrates test data displayed on a self-organizing map based ona 9×18 direct kernel self-organizing map (DK-SOM) in wrap-around mode.

FIG. 8 illustrates results for the prediction of ischemia usingdifferent techniques on the test data set.

FIG. 9 is an operation schematic for direct kernel methods as a datapre-processing step.

FIG. 10 is a flow diagram illustrating data pre-processing with kernelcentering for direct kernel methods.

FIG. 11 is typical structure of a self-organizing map (SOM).

FIG. 12 is a list of techniques disclosed herein for the automaticclassification of cardiograph data.

DETAILED DESCRIPTION

This disclosure describes the use of direct-kernel methods and supportvector machines for pattern recognition in magnetocardiography (MCG)that measures magnetic fields emitted by the electrophysiologicalactivity of the human heart. A SQUID-based measuring device for MCG thatcan be used in regular hospital rooms (not specially shielded fromelectromagnetic interference) is presently under separate development.The operation of the system is computer-controlled and largelyautomated. Proprietary software is used for precise 24-bit control anddata acquisition followed by filtering, averaging, electric/magneticactivity localization, heart current reconstruction, and derivation ofdiagnostic scores.

The interpretation of MCG recordings remains a challenge. Hence, thisdisclosure considers the methods to automate interpretation of MCGmeasurements to minimize human input for the analysis. Testing hasfocused on detecting ischemia, a condition arising in many common heartdiseases that may result in heart attack, the leading cause of death inthe United States, but this exemplary, not limiting.

Scientifically, this disclosure considers a two-class separation problem(diseased heart vs. healthy heart) where the number of descriptors (datapoints) exceeds the number of datasets. Hence, this disclosure focusesnow on two tasks for the solution of this problem.

The first question to be answered is whether the problem is linear ornon-linear because this determines the class of possible candidatefunctions that can solve the problem (known as “hypotheses” or “machinelearning techniques”). Our goal is to keep the separation process itselflinear and encode non-linearities, if any, in the pre-processing. Thelatter can be achieved by applying a (non-linear) kernel transformationto the data prior to performing the actual machine learning (we refer totechniques that operate on kernel transformed data as “non-linear”techniques). Hence, if the data contain non-linearities, purely linearmethods will show an inferior performance compared to non-lineartechniques.

The second objective is to find (or develop) machine learning techniquesthat actually solve the separation problem. The focus here is not somuch on finding the best solution but a class of techniques that performequally well on the data. This helps to build confidence in the chosenmodels and their generalization capabilities (the ability of ahypothesis to correctly classify data not in the training set is knownas its “generalization”). It is easy to develop a model that performsoptimally on the training data but cannot predict unseen data (thephenomenon often referred to as overtraining). It is very difficult,however, to build (and tune) a model with good prediction based on onlyfew data.

We start with a discussion of data acquisition and preprocessing.Particularly, we discuss what kind of preprocessing is suitable todifferent learning methods. Thereafter we present the core results: thecomparison of performance of different machine learning techniques forour problem, and methodologies for assessment of prediction quality andfor the regularization parameter selection. Thereafter we discussfeature selection.

Data Acquisition and Pre-Processing

MCG data are acquired at 36 locations above the torso by making foursequential measurements in mutually adjacent positions. In each positionthe nine sensors measure the cardiac magnetic field for 90 seconds usinga sampling rate of 1000 Hz leading to 36 individual time series. Fordiagnosis of ischemia, a bandwidth of 0.5 Hz to 20 Hz is needed, so ahardware low pass filter at 100 Hz using 6th-order Bessel filtercharacteristics is applied, followed by an additional digital low passfilter at 20 Hz using the same characteristics, but a higher order. Toeliminate remaining stochastic noise components, the complete timeseries is averaged using the maximum of the R peak of the cardiac cycleas a trigger point. For automatic classification, we used data from atime window between the J point and T peak [5] of the cardiac cycle inwhich values for 32 evenly spaced points were interpolated from themeasured data. The training data consist of 73 cases that were easy toclassify visually by trained experts. The testing was done on a set of36 cases that included patients whose magnetocardiograms misled orconfused trained experts doing visual classification.

Data were preprocessed in this case by first subtracting the bias fromeach signal. Then, we investigated the most effective pre-processing forour multi-variate time-series signals, including local, vertical,horizontal and two-dimensional (global) Mahanalobis scaling, and wavelettransforms. An important consideration was preservation of datalocality, which was achieved by applying to each signal the Daubechies-4wavelet transform [3]. It was chosen, because of the relatively smallset of data (32) in each of the interpolated time signals. Only SOM andK-PLS methods that observe data locality in input did not require thistransformation. Next, we Mahalanobis scaled the data, first on all 36signals and then (for all except SOM based methods) vertically. Atypical dataset for 36 signals that are interpolated to 32 equallyspaced points in the ST segment [5] and after Mahalanobis scaling oneach of the individual signals is shown in FIG. 1.

Predicitive Modeling for MCG Data Classification

The aim of machine learning is to delegate some basics of intelligentdecision making to computers. In its current form a significant part ofmachine learning is based on the development of robust classification,regression tools and feature selection methods.

The ultimate aim of machine learning in the context of cardiac diagnosisis to be able to identify meaningful features that can explain the modeland allow the formulation of expert rules that have transparency.

A key ingredient of machine learning is the prevention of overtraining.The concept of Tikhonov regularization is a powerful concept in machinelearning for doing so. A second issue in machine learning is the needfor building reliable nonlinear methods. Support vector machines (SVMs)and other kernel-based methods, such as kernel principal componentanalysis, kernel ridge regression, and partial least squares arepowerful methods for incorporating non-linearity and regularization inmachine learning methods.

Current challenges in machine learning are in large problems with morefeatures than data, problems with lots of uncertainty and noise in thedata, and non-ordinal multi-class classification problems with mixturemodes.

The need for proper pre-processing is highly domain dependent, butexploring different pre-processing methods, and incorporating domainexpertise in this pre-processing stage is a key ingredient to makingmachine learning methods work.

Our purpose is to identify not only “the best” machine learning methodbut also a class of techniques that perform equally well on the data.Therefore, we consider SVMs, a prime tool in the machine learningcommunity. We also use other kernel based methods that might be easierto tune or easier to implement in hardware than SVMs, but were expectedto perform comparably to SVMs.

One key for successful machine learning lies in the pre-processing ofthe data. Many different pre-processing scenarios are worthy ofconsideration. We distinguish among four types of preprocessing asfollows:

-   1. Normalization: this is needed to make the data comparable. It    usually means that the data are scaled and de-biased. However, one    has many choices here.-   2. Localization of information: by localization we mean applying a    transform that rearranges the data such that the coefficients, which    contain most of the information, are presented first. One prominent    example is the Wavelet Transform that even preserves locality of    information.-   3. Feature selection: this usually operates on already transformed    data. It means that coefficients that either contain no or little    information are cut out to reduce the dimensionality of the input    domain. This is particularly useful to speed up the learning.-   4. Kernel transformation: The kernel transformation is an elegant    way to make a regression model nonlinear. A kernel is a matrix    containing similarity measures for a dataset: either between the    data of the dataset itself, or with other data (e.g., support    vectors [2]).

It is clear that this offers a variety of combinations of approaches forimproving cardiogram classification.

Turning first to normalization, It is a common procedure in machinelearning to center all the descriptors and to bring them to a unityvariance. The same process is then applied to the response. Thisprocedure of centering and variance normalization is known asMahalanobis scaling. While Mahalanobis scaling is not the only way topre-process the data, it is probably the most general and the mostrobust way to do pre-processing that applies well across the board. Ifwe represent a feature vector as {right arrow over (z)}, Mahalanobisscaling will result in a rescaled feature vector {right arrow over (z)}′and can be summarized as:

$\begin{matrix}{{\overset{\_}{z}}^{\prime} = \frac{\overset{\_}{z} - \overset{\_}{z}}{{std}( \overset{\_}{z} )}} & (1)\end{matrix}$

where {right arrow over (z)} represents the average value and std({rightarrow over (z)}) represents the standard deviation for attribute {rightarrow over (z)}.

We will refer to “horizontal Mahalanobis scaling” when the 36 timeseries are scaled individually (along the time axis), to “verticalMahalanobis scaling” when all 36 points at one instance of time arescaled, and to “global Mahalanobis scaling” when all 36 points at all 32time instances are scaled.

We turn next to localization. Applying a Wavelet transform [10] helps tolocalize “hot spots” of information on the one hand and “cold areas”that don't contribute to the signal on the other hand. The property thatmakes the Wavelet transform more suitable than a Fourier transform isthat individual wavelet functions are localized in space. Fourier sineand cosine functions are not. Wavelet transforms do not have a singleset of basis functions but an infinite set of possible basis functions.

Dilations and translations of the “Mother function,” or “analyzingwavelet” Φ(x) define an orthogonal basis, also known as the waveletbasis:

$\begin{matrix}{{\Phi ( {s,l} )} = {2^{\frac{- s}{s}}{\Phi ( {{2^{- s}x} - l} )}}} & (2)\end{matrix}$

The variables s and l are integers that scale and dilate the motherfunction Φ(x) to generate wavelets, such as a Daubechies wavelet family.The scale index s indicates the wavelet's width, and the location indexl gives its position. Notice that the mother functions are rescaled, or“dilated” by powers of two, and translated by integers. What makeswavelet bases especially interesting is the self-similarity caused bythe scales and dilations. Once we know about the mother functions, weknow everything about the basis.

To span our data domain at different resolutions, the analyzing waveletis used in a scaling equation:

$\begin{matrix}{{W(x)} = {\sum\limits_{k = {- 1}}^{N - 2}\; {( {- 1} )^{k}c_{k + 1}{\Phi ( {{2\; x} + k} )}}}} & (3)\end{matrix}$

where W(x) is the scaling function for the mother function Φ(x), andc_(k) are the wavelet coefficients. The wavelet coefficients mustsatisfy linear and quadratic constraints of the form

$\begin{matrix}{{{\sum\limits_{k = 0}^{N - 1}\; c_{k}} = 2},{{\sum\limits_{k = 0}^{N - 1}\; {c_{k}c_{k + {2\; l}}}} = {2\delta_{l,0}}}} & (4)\end{matrix}$

where δ is the delta function and l is the location index.

One of the most useful features of wavelets is the ease with which ascientist can choose the defining coefficients for a given waveletsystem to be adapted for a given problem. In Daubechies' paper [3], shedeveloped specific families of wavelet systems that were very good forrepresenting polynomial behavior. For MCG time series, the so-called“Daubechies 4” wavelet showed optimal performance.

We turn now to the kernel transform. The kernel transform and its tuningis an important component of the ability to improve cardiogramclassification. Therefore, we will explain this concept in more detailand pinpoint some major problems which are commonly overlooked whenapplying kernel transforms.

The kernel transformation is an elegant way to make a regression modelnonlinear. The kernel transformation goes back at least to the early1900's, when Hilbert introduced kernels in the mathematical literature.A kernel is a matrix containing similarity measures for a dataset:either between the data of the dataset itself, or with other data. Aclassical use of a kernel is as the correlation matrix in principalcomponent analysis, where the feature kernel contains linear similaritymeasures between attributes or features. In support vector machines, thekernel entries are similarity measures between data rather than featuresand these similarity measures are usually nonlinear. There are manypossible nonlinear similarity measures, but in order to bemathematically tractable the kernel has to satisfy certain conditions,the so-called Mercer conditions [2, 11, 15].

$\begin{matrix}{{\overset{rightarrow}{K}}_{m} = \begin{bmatrix}k_{11} & k_{12} & \ldots & k_{1\; n} \\k_{21} & k_{22} & \ldots & k_{2\; n} \\\; & {\; \ldots} & \; & \; \\k_{n\; 1} & k_{n\; 2} & \ldots & k_{nn}\end{bmatrix}} & (5)\end{matrix}$

The expression above, introduces the general structure for the datakernel matrix, {right arrow over (K)}_(nm), for n data. The kernelmatrix is a symmetrical matrix where each entry contains a (linear ornonlinear) similarity between two data vectors. There are many differentpossibilities for defining similarity metrics such as the dot product,which is a linear similarity measure and the Radial Basis Functionkernel or RBF kernel, which is a nonlinear similarity measure. The RBFkernel is the most widely used nonlinear kernel and its entries aredefined by

$\begin{matrix}{k_{ij} \equiv ^{- \frac{{{{\overset{arrow}{x}}_{j} - {\overset{arrow}{x}}_{l}}}_{2}}{2\sigma^{2}}}} & (6)\end{matrix}$

Note that in the kernel definition above, the kernel entry contains theEuclidean distance between data points, which is a dissimilarity (ratherthan a similarity) measure, in a negative exponential. The negativeexponential also contains a free parameter, σ, which is the Parzenwindow width for the RBF kernel. The proper choice for selecting theParzen window is usually determined by an additional tuning, also calledhyper-tuning, on an external validation set. The precise choice for σ isnot crucial, there usually is a relatively broad range for the choicefor σ for which the model quality is stable.

The kernel transformation is applied here as a data transformation in aseparate pre-processing stage. We actually replace the data by anonlinear data kernel and apply a traditional linear predictive model.Methods where a traditional linear algorithm is used on a nonlinearkernel transform of the data are introduced and defined here as “directkernel methods.” The elegance and advantage of such a direct kernelmethod is that the nonlinear aspects of the problem are captured in thekernel and are transparent to the applied algorithm.

One could also represent the kernel transformation in a neural networktype of flow diagram and the first hidden layer would now yield thekernel-transformed data, and the weights in the first layer would bejust the descriptors of the training data. The second layer contains theweights that can be calculated with a numerical method, such as kernelridge regression (see FIG. 9). When a radial basis function kernel isused, this type of neural network would look very similar to a radialbasis function neural network [17,18], except that the weights in thesecond layer are calculated differently.

It is also important to deal with bias by centering the Kernel. Lookingat the general prediction equation

{circumflex over (y)} _(n) =X _(nm) {right arrow over (w)} _(m)  (7)

in which the weight vector {right arrow over (w)}_(m) is applied to thedata matrix X_(nm) to reach the (predicted) output ŷ _(n), there is noconstant offset term. It turns out that for data that are centered thisoffset term (“bias”) is always zero and does not have to be includedexplicitly. Rather than applying equation 7, a more general predictivemodel that includes this bias can be written as:

ŷ _(n) =X _(nm) {right arrow over (w)} _(m) +b  (8)

where b is the bias term. Because we make it a practice to center thedata first by Mahalanobis scaling, this bias term is zero and can beignored.

When dealing with kernels, the situation is more complex, as they needsome type of bias. We will give only a recipe here, that works well inpractice, and refer to the literature for more details on why [11, 12,16, 19]. Even when the data were Mahalanobis-scaled before applying akernel transform, the kernel still needs some type of centering to beable to omit the bias term in the prediction model. A straightforwardway for centering the kernel is to subtract the average from each columnof the training data kernel, and store this average for later recall,when centering the test kernel. A second step for centering the kernelis going through the newly-obtained vertically centered kernel again,this time row by row, and subtracting the row average form eachhorizontal row.

The kernel of the test data needs to be centered in a consistent way,following a similar procedure. In this case, the stored column averagesfrom the kernel of the training data will be used for the verticalcentering of the kernel of the test data. This vertically-centered testkernel is then centered horizontally, i.e., for each row, the average ofthe vertically centered test kernel is calculated, and each horizontalentry of the vertically centered test kernel is substituted by thatentry minus the row average.

The advantages of this algorithm for centering kernels as disclosedabove is that it applies on rectangular data kernels as well. The flowchart for pre-processing the data, applying a kernel transform on thisdata, and then centering the kernel for the training data, validationdata, and test data is shown in FIG. 10.

Both unsupervised and supervised learning methods were investigated. Forunsupervised learning, Direct Kernel (DK)-SOMs were used, since SOMs areoften applied for novelty detection and automated clustering. The DK-SOMused has a 9×18 hexagonal grid with unwrapped edges. For supervisedlearning, four kernel-based regression algorithms were used: classicalSupport Vector Machines effective in extracting relevant parameters fromcomplex data spaces; kernel partial least square K-PLS, as proposed byRosipal [10]; direct kernel partial least square (DK-PLS); andLeast-Squares Support Vector Machines (i.e., LS-SVM, also known askernel ridge regression).

Support Vector Machines or SVMs have proven to be formidable machinelearning tools because of their efficiency, model flexibility,predictive power, and theoretical transparency [2,11,15]. While thenonlinear properties of SVMs can be exclusively attributed to the kerneltransformation, other methods, such as self-organizing maps or SOMs [9],are inherently nonlinear because they incorporate variousneighborhood-based manipulations. Unlike SVMs, the prime use for SOMs isoften as a visualization tool [4] for revealing the underlyingsimilarity/cluster structure of high-dimensional data on atwo-dimensional map, rather than for regression or classificationpredictions.

The Analyze/StripMiner software package was used, developed in-house forthe analysis [14], but SVMLib [1] for the SVM model was also used. Usingthe training set, the values for the parameters in DK-SOM, SVM, DK-PLSand LS-SVM were optimized before testing. The results are similar to thequality of classification achieved by the trained experts and similarfor all tested methods, even though they use different datapreprocessing. This is important because it indicates that there was noover-training in any of the tested methods. The agreement betweenDK-PLS, SVMLib, and LS-SVM is particularly good, and there are nonoticeable differences between these methods on these data. The resultsare shown in FIGS. 2 and 3. FIG. 2 lists the number of correctlyclassified patterns and the number of misses for the negative andpositive cases. FIG. 3 provides additional mesaures of quality ofprediction. Further results are shown in FIG. 8. In FIG. 8, RMSE denotesthe root mean square error (the smaller the better) and CC [%] means thepercentage of correctly classified cases. The best result was obtainedfor the DK-PLS method, which also showed the best robustness. Thisalready outperforms the predictive accuracy of three standard testscombined (ECG, ECHO, and Troponin I), which is 58% for these patients.

After tuning, the Parzen window width σ for SVM was chosen as 10. Theregularization parameter, C, in SVMLib was set to 1/λ as suggested in[10]. Based on experience with other applications [14] and scalingexperiments, ridge parameter λ was determined from the followingequation, for an n data kernel:

$\begin{matrix}{\lambda = {\min \{ {1;( \frac{n}{1500} )^{\frac{3}{2}}} \}}} & (9)\end{matrix}$

More generally, empirically, we have found that λ is proportional to thenumber of data n to the power of 3/2.

The agreement between the direct kernel methods (DK-PLS and LS-SVM),K-PLS, and the traditional kernel-based SVM (SVMLib) indicates anear-optimal choice for the ridge parameter resulting from this formula.

Turning now to metrics for assessing the model quality, for a regressionproblem, another way to capture the error is by the Root Mean SquareError index or RMSE, which is defined as the average value of thesquared error (either for the training set or the test set) accordingto:

$\begin{matrix}{{RMSE} = \sqrt{\frac{1}{n}{\sum\limits_{i}\; ( {{\hat{y}}_{i} - y_{i}} )^{2}}}} & (10)\end{matrix}$

While the root mean square error is an efficient way to compare theperformance of different prediction methods on the same data, it is notan absolute metric in the sense that the RMSE will depend on how theresponse for the data was scaled. In order to overcome this handicap,additional error measures were also used which are less dependent on thescaling and magnitude of the response value. A first metric used forassessing the quality of a trained model is r², which is defined as thesquared coefficient of correlation between target values and predictionsfor the response according to:

$\begin{matrix}{r^{2} = \frac{\sum\limits_{i = 1}^{n_{train}}\; {( {{\hat{y}}_{i} - \overset{\_}{y}} )( {y_{i} - \overset{\_}{y}} )}}{\sqrt{\sum\limits_{i = 1}^{n_{train}}\; ( {{\hat{y}}_{i} - \overset{\_}{y}} )^{2}}\sqrt{\sum\limits_{i = 1}^{n_{train}}\; ( {y_{i} - \overset{\_}{y}} )^{2}}}} & (11)\end{matrix}$

where n_(train) represents the number of data points in the trainingset. r² takes values between zero and unity, and the higher the r²value, the better the model. An obvious drawback of using r² forassessing the model quality is that it only expresses a linearcorrelation, indicating how well the predictions follow a line if ŷ isplotted as function of y. While one would expect a nearly perfect modelwhen r² is unity, this is not always the case. A second, and morepowerful measure to assess the quality of a trained model is theso-called “Press r squared”, or R², often used in chemometric modeling[6], where R² is defined as [7]:

$\begin{matrix}{R^{2} = {1 - \frac{\sum\limits_{i = 1}^{n_{train}}\; ( {y_{i} - {\hat{y}}_{i}} )^{2}}{\sum\limits_{i = 1}^{n_{train}}\; ( {y_{i} - \overset{\_}{y}} )^{2}}}} & (12)\end{matrix}$

R² is considered a better measure than r², because it accounts for theresidual error as well. Just like r², R² ranges between zero and unity,and higher the value for R², better the model. The R² metric isgenerally smaller than r². For large datasets, R² tends to converge tor², and the comparison between r² and R² for such data often revealshidden biases.

For assessing the quality of the validation set or a test set, weintroduce similar metrics, q² and Q², where q² and Q² are defined as1−r² and 1−R², respectively, for the data in the test set. For a modelthat perfectly predicts on the test data we would expect q² and Q² to bezero. The reason for introducing metrics which are symmetric between thetraining set and the test set is actually to avoid confusion. Q² and q²values apply to a validation set or a test set, and one would expectthese values to be quite low in order to have a good predictive model.R² and r² values apply to training data, and it is easy to notice thatif the predictions are close to actual values, they both are close tounity. Hence, any of them significantly different from 1 indicates amodel with poor predictive ability.

Linear methods, such as partial-least squares, result in inferiorpredictive models as compared to the kernel methods. For K-PLS andDK-PLS, 5 latent variables were chosen, but the results were notcritically dependent on the exact choice of the number of latentvariables. Direct Kernel Principal Component Analysis (DK-PCA) was alsotried, which is the direct kernel version of K-PCA [11-12,16], but theresults were more sensitive to the choice of the number of principalcomponents and not as good as the ones obtained using other directkernel methods.

Typical prediction results for the magnetocardiogram data based onwavelet transformed data and DK-PLS are shown in FIG. 4. It can be seenfrom this figure that six data points are misclassified altogether inthe predictions (one healthy or negative case and five ischemia cases).These cases were also difficult to identify correctly for the trainedexpert, based on a 2-D visual display of the time-varying magneticfield, obtained by proprietary methods.

For medical data, it is often important to be able to make a trade-offbetween false negative and false-positive cases, or between sensitivityand specificity (which are different metrics related to false positivesand false negatives). In machine-learning methods, such a trade-off caneasily be accomplished by changing the threshold for interpreting theclassification. For example, in FIG. 4, rather than using zero as thediscrimination value, one could shift the discrimination thresholdtowards a more desirable level, hereby influencing the falsepositive/false negative ratio.

A summary of all possible outcomes of such changes in the discriminationvalue can be displayed in an ROC curve, as shown in FIG. 5 for the abovecase. The concept of ROC curves (or Receiver Operator Characteristics)originated from the early development of the radar in the 1940's foridentifying airplanes and is summarized in [13].

FIG. 6 displays a projection of 73 training data, based on (left) DirectKernel Principal Component Analysis (DK-PCA), and (right) Direct KernelPLS (DK-PLS). Diseased cases are shown as filled circles. The right sideFIG. 6 shows a clearer separation and wider margin between differentclasses, based on the first two components for DK-PLS as compared toresults of DK-PCA that are shown in the left side of FIG. 6. The testdata, originally shown on these pharmaplots as dark and light crosses,shows an excellent separation between healthy and diseased cases forboth methods.

A typical 9×18 self-organizing map on a hexagonal grid in wrap-aroundmode, based on the direct kernel SOM, is shown in FIG. 7. Thewrap-around mode means that the left and right boundaries (and also thetop and bottom boundaries) flow into each other, and that the map is anunfurling of a toroidal projection. The dark hexagonals indicatediseased cases, while the light hexagonals indicate healthy cases. Fullycolored hexagons indicate the positions for the training data, while thewhite and dark-shaded numbers are the pattern identifiers for healthyand diseased test cases. Most misclassifications actually occur onboundary regions in the map. The cells in the map are colored bysemi-supervised learning, i.e., each data vector, containing 36×32 or1152 features, is augmented by an additional field that indicates thecolor. The color entry in the data vectors are updated in a similar wayas for the weight vectors, but they are not used to calculate thedistance metrics for determining the winning cell. The resulting mapsfor a regular SOM implementation are very similar to those obtained withthe direct kernel DK-SOM. The execution time for generating DK-SOM on a128 MHz Pentium III computer was 28 seconds, rather than 960 secondsrequired for generating the regular SOM, because the data dimensionalitydropped to effectively 73 (the number of training data) from theoriginal 1152, after the kernel transformation on the data. Thefine-tuning for the SOM and DK-SOM was done in a supervised mode withlearning vector quantization [9]. While the results based on SOM andDK-SOM are still excellent, they are not as good as those obtained withthe other kernel-based methods (SVMLib, LS-SVM, and K-PLS).

Feature Selection

The results presented in the previous section were obtained using all1152 (36×32) descriptors. It would be most informative to the domainexpert, if one could identify where exactly in the time or waveletsignals and for which of the 36 magnetocardiogram signals that weremeasured at different positions for each patient the most importantinformation necessary for good binary classification is located. Suchinformation can be derived by feature selection.

Feature selection, i.e., the identification of the most important inputparameters for the data vector, can proceed in two different ways: thefiltering mode and the wrap-around mode. Ordinarily, these twoapproaches are employed separately from one another; however, they mayalso be combined within the scope of this disclosure and its associatedclaims.

In the filtering mode, features are eliminated based on a prescribed,and generally unsupervised procedure. An example of such a procedurecould be the elimination of descriptor columns that contain four σoutliers, as is often the case in PLS applications for chemometrics. Itis also common to drop “cousin” descriptors in a filtering mode, i.e.,features that show more than 95% correlation with another descriptor.Depending on the modeling method, it is often common practice to dropthe cousin descriptors and only retain the descriptors that (i) eithershow the highest correlation with the response variable, or (ii) havethe clearest domain transparency to the domain expert for explaining themodel.

The second mode of feature selection is based on the wrap-around mode.One wants to retain only the most relevant features necessary to have agood predictive model. Often, the modeling quality improves after theproper selection of the optimal feature subset. Determining the rightsubset of features can proceed based on different concepts, and theresulting subset of features is often dependent on the modeling method.Feature selection in a wrap-around mode generally proceeds by using atraining set and a validation set, and the validation set is used toconfirm that the model is not over-trained by selecting a spurious setof descriptors. Two generally applicable methods for feature selectionsare based on the use of genetic algorithms and sensitivity analysis.

The idea with the genetic algorithm approach is to be able to obtain anoptimal subset of features from the training set, showing a goodperformance on the validation set as well.

The concept of sensitivity analysis [8] exploits the saliency offeatures, i.e., once a predictive model has been built, the model isused for the average value of each descriptor, and the descriptors aretweaked, one-at-a time between a minimum and maximum value. Thesensitivity for a descriptor is the change in predicted response. Thepremise is that when the sensitivity for a descriptor is low, it isprobably not an essential descriptor for making a good model. A few ofthe least sensitive features can be dropped during one iteration step,and the procedure of sensitivity analysis is repeated many times until anear optimal set of features is retained. Both the genetic algorithmapproach and the sensitivity analysis approach are true soft computingmethods and require quite a few heuristics and experience. The advantageof both approaches is that the genetic algorithm and sensitivityapproach are general methods that do not depend on the specific modelingmethod.

Further Comments Regarding Machine Learning

Rather than reviewing here all available machine learning techniques, wefirst address the question why we didn't simply use support vectormachines (SVMs), the state-of-the-art solution for both linear andnon-linear problems. Scientifically, as noted earlier, the goal is tofind a class of techniques that performs equally well on the givenproblem to ensure a stable solution. Within this class, the optimalmodel is then the one that is easiest to tune and that executes fastest.Comparing these models against SVMs as the standard helps verify theperformance of any newly developed technique.

Regarding supervised learning, we give here a short description of theso called machine learning paradox in supervised learning, which is thereason for the multitude of models that were developed to find a way outof the dilemma.

It is customary to denote the data matrix as X_(Nm) and the responsevector as {right arrow over (y)}_(N), assuming that there are N datapoints and m descriptive features in the dataset. We would like to infer{right arrow over (y)}_(N) from X_(Nm) by induction, denoted as X_(nm)

{right arrow over (y)}_(N), in such a way that our inference modelderived from n training data points, but also does a good job on theout-of-sample data (i.e., N-n validation data and test data points). Inother words, we aim at building a linear predictive model of the type:

{circumflex over (y)} _(n) =X _(nm) {right arrow over (w)} _(m)  (13)

This equation assumes a known weight vector {right arrow over (w)}_(m)that has to be determined in a prior step, the actual learning tosatisfy, in the best case, equation:

X _(nm) {right arrow over (w)} _(m) ={right arrow over (y)} _(n)  (14)

Here, X_(nm) are training data and {right arrow over (y)}_(n) denotesthe known answers (“labels”).

Note that the data matrix is generally not symmetric. If that were thecase, it would be straightforward to come up with an answer by using theinverse of the data matrix. We will therefore apply the pseudo-inversetransformation, which will generally not lead to precise predictions fory, but will predict y in a way that is optimal in a least-squares sense.The pseudo-inverse solution for the weight vector is illustrated below:

(X _(nm) ^(T) X _(nm)){right arrow over (w)} _(m) =X _(nm) ^(T) {rightarrow over (y)} _(n)

(X _(nm) ^(T) X _(nm))⁻¹(X _(nm) ^(T) X _(nm)){right arrow over (w)}_(m)=(X _(nm) ^(T) X _(nm))⁻¹ X _(nm) ^(T) {right arrow over (y)} _(n)

{right arrow over (w)} _(m)=(X _(nm) ^(T) X _(nm))⁻¹ X _(nm) ^(T) {rightarrow over (y)} _(n)

{right arrow over (w)} _(m)(K _(F))_(mn) ⁻¹ X _(nm) ^(T) {right arrowover (y)} _(n)  (15)

K_(F)=X^(T) _(mn)X_(nm) is the so-called “feature kernel matrix” and thereason for the machine learning paradox: Learning occurs only because ofredundancies in the features—but then, K_(F) is ill-conditioned (rankdeficient). As indicated before, there are several ways to resolve theparadox:

-   1. By fixing the rank deficiency of K_(F) with principal components    (calculating the eigenvectors of the feature kernel) [18]-   2. By regularization: use K_(F)+λI instead of K_(F) (ridge    regression) [17, 20-23]-   3. By local learning

We used four kernel-based regression algorithms: classical SupportVector Machines effective in extracting relevant parameters from complexdata spaces [2, 12, 15], kernel partial least squares (K-PLS), asproposed by Rosipal[10], direct kernel partial least squares (DK-PLS),and Least-Squares Support Vector Machines (i.e., LS-SVM, also known askernel ridge regression [24-28]). Additionally, we tested direct kernelprincipal component analysis (DK-PCA).

Partial least squares (PLS) is one of the standard analysis methods inQSAR and chemo metrics [29]. Kernel PLS (K-PLS) is a recently developednonlinear version of PLS, introduced by Rosipal and Trejo [10]. K-PLS isfunctionally equivalent to SVMs but unlike SVMs, results turn out to bea bit more stable. K-PLS is currently used to predict binding affinitiesto human serum albumin.

In the work underlying this disclosure, we improved K-PLS to DK-PLS andmade use of the earlier experiences developing the code for K-PLS,DK-PLS, DK-PCA, and LS-SVM in the Analyze/Stripminer program [14]. Thedifference between K-PLS and DK-PLS is that the feature (data) kernelmatrix is used in K methods while this matrix is replaced by the(non-linear) kernel-transformed matrix in DK methods. For calculatingthe matrix inverse we applied Møller's scaled conjugate gradient method[30], which is implemented in the Analyze/Stripminer program.

Turning to unsupervised learning, we observe that the SOM [9, 17, 31-36]is an unsupervised learning neural network developed by Kohonen. The SOMis an iterative method based on competitive learning. It provides amapping from a high-dimensional input data space into the lowerdimensional output map, usually a one or two-dimensional map, see FIG.11. Components (or data points) are loaded into the input layer and theSOM is trained using a competitive learning algorithm [4]. The weightsare updated according to

{right arrow over (w)} _(m) ^(new)=(1−α){right arrow over (w)} _(m)^(old) +α{right arrow over (x)} _(m),

where α is the learning rate parameter. As the result of the learningthe input data will be mapped onto the “winning” neuron. As a result ofthis process, the SOM is often used for dimension reduction andclustering. Moreover, a distinguishing feature of the SOM is that itpreserves the topology of the input data from the high-dimensional inputspace onto the output map in such a way that relative distances betweeninput data are more or less preserved [38]. Input data points that arelocated close to each other in the input space, are mapped to nearbyneurons on the output map. SOM-based visualization methods are versatiletools for data exploration. They are used for clustering, correlationdetection and projection of data [4,39].

The traditional SOM is a method based on the projection of thehigh-dimensional input data onto the lower-dimensional output map.Disclosed here, is a new kernel based SOM. The kernel SOM is now trainedon the kernel representation of the data rather than on the originaldata. Using kernel transformed data here is not so much to “discover”non-linearities in the data because SOMs are inherently non-linear butto increase (learning) speed because the kernel has fewer effectivefeatures.

In summary, we have used and developed a set of machine learning toolsthat is presented in FIG. 12.

CONCLUSION

The binary classification of MCG data represents a challenging problem,but its solution is important if MCG is to succeed in clinical practice.Applying existing machine learning techniques such as SOM and SVM to MCGdate results in a predictive accuracy of 74%. Very significantimprovement is achieved by transforming the data first into the waveletdomain and additionally applying a kernel transform on waveletcoefficients, and by even applying the kernel transform alone withoutthe wavelet transform. This increased the predictive accuracy to 83%.

The agreement of the results between kernel PLS (K-PLS) as proposed byRosipal [10], direct kernel PLS (DK-PLS), support vector machine(SVMLib), and least square SVM (LS-SVM) is generally excellent. In thiscase, DK-PLS gave a superior performance, but the differences amongkernel-based methods are not significant. This excellent agreement showsthe robustness of the direct kernel methods. It could only be achievedif the selection of the ridge parameter by Equation (1) was nearlyoptimal. This selection defines also the regularization parameter, C, insupport vector machines, when C is taken as 1/λ.

The obtained results are meaningful for the medical community. DK-PLSwas used to reach a sensitivity of 92% and a specificity of 75% for thedetection of ischemia defined by coronary angiography. It is of noticethat MCG is a purely functional tool that is sensitive for abnormalitiesin the electrophysiology of the heart and, therefore, can only diagnosethe effect of a disease. The gold standard (coronary angiography),however, is a purely anatomical tool and diagnoses one cause of ischemicheart disease. Since MCG detects abnormalities that are not visible tothe gold standard, it will always produce “false positives”, whichexplains the comparatively low specificity in this application.

Note that the kernel transformation is applied here as a datatransformation in a separate pre-processing stage. The data is actuallyreplaced by a nonlinear data kernel and apply a traditional linearpredictive model is then applied. Methods where a traditional linearalgorithm is used on a nonlinear kernel transform of the data definedwhat is referred to here as “direct kernel methods.” The elegance andadvantage of such a direct kernel method is that the nonlinear aspectsof the problem are captured in the kernel and are transparent to theapplied algorithm.

While the kernels discussed herein are Gaussian in nature, this isexemplary, not limiting. For example, not limitation, so-called splinekernels may also be used and are regarded within the scope of thisdisclosure.

While only certain preferred features of the invention have beenillustrated and described, many modifications, changes and substitutionswill occur to those skilled in the art. It is, therefore, to beunderstood that the appended claims are intended to cover all suchmodifications and changes as fall within the true spirit of theinvention.

REFERENCES

-   [1] C.-C. Chang and C.-J. Lin, LibSVM, OSU, see    http://www.csie.ntu.edu.tw/˜cjlin/libsvmSVMLib.-   [2] N. Cristianini and J. Shawe-Taylor [2000] Support Vector    Machines and Other Kernel-Based Learning Methods, Cambridge    University Press.-   [3] I. Daubechies [1992], Ten Lectures on Wavelets, Siam,    Philadelphia, Pa.-   [4] G. Deboeck and T. Kohonen (Eds.) [1998] Visual Explorations in    Finance with Self-Organizing Maps, Springer.-   [5] V. Froelicher, K. Shetler, and E. Ashley [2002] “Better    Decisions through Science: Exercise Testing Scores.” Progress in    Cardiovascular Diseases, Vol. 44(5), pp. 385-414.-   [6] A. Golbraikh and A. Tropsha [2002] “Beware of q² !” Journal of    Molecular Graphics and Modelling, Vol 20, pp. 269-276.-   [7] R. A. Johnson and D. W. Wichern [2000] Applied Multivariate    Statistical Analysis, 2 ed., Prentice Hall.-   [8] R. H. Kewley, and M. J. Embrechts [2000] “Data Strip Mining for    the Virtual Design of Pharmaceuticals with Neural Networks,” IEEE    Transactions on Neural Networks, Vol. 11 (3), pp. 668-679.-   [9] T. Kohonen [1997] Self-Organizing Maps, 2^(nd) Edition,    Springer.-   [10] R. Rosipal and L. J. Trejo [2001] “Kernel Partial Least Squares    Regression in Reproducing Kernel Hilbert Spaces,” Journal of Machine    Learning Research, Vol. 2, pp. 97-128.-   [11] B. Schölkopf and A. J. Smola [2002] Learning with Kernels, MIT    Press.-   [12] B. Schölkopf, A. Smola, and K-R Müller [1998] “Nonlinear    Component Analysis as a Kernel Eigenvalue Problem,” Neural    Computation, Vol. 10, 1299-1319, 1998.-   [13] J. A. Swets, R. M. Dawes, and J. Monahan[2000] “Better    Decisions through Science,” Scientific American, pp. 82-87.-   [14] The Analyze/StripMiner, the description and the code are    available at http://www.drugmining.com.-   [15] V. Vapnik [1998] Statistical Learning Theory, John Wiley &    Sons.-   [16] W. Wu, D. L. Massarat and S. de Jong [1997] “The Kernel PCA    Algorithm for Wide Data. Part II: Fast Cross-Validation and    Application in Classification of NIR Data,” Chemometrics and    Intelligent Laboratory Systems, Vol. 37, pp. 271-280.-   [17] A. E. Hoerl, and R. W. Kennard [1970] “Ridge Regression: Biased    Estimation for Non-Orthogonal Problems,” Technometrics, Vol. 12, pp.    69-82.-   [18] J. Principe, N. R. Euliano, and W. C. Lefebre [2000] Neural and    Adaptive Systems: Fundamentals through Simulations, John Wiley &    Sons, Inc.-   [19] W. Wu, D. L. Massarat and S. de Jong [1997] “The Kernel PCA    Algorithm for Wide Data. Part I: Theory and Algorithms,”    Chemometrics and Intelligent Laboratory Systems, Vol. 36, pp.    165-172.-   [20] Vladimir Cherkassky and Filip Mulier [1998] Learning from Data:    Concepts, Theory, and Methods, John Wiley & Sons, Inc.-   [21] S. Haykin [1999] Neural Networks: A Comprehensive Foundation    (2^(nd) Ed.), Prentice Hall.-   [22] A. N. Tikhonov [1963] “On Solving Incorrectly Posed Problems    and Method of Regularization,” Doklady Akademii Nauk USSR, Vol. 151,    pp. 501-504.-   [23] A. N. Tikhonov and V. Y. Arsenin [1977] Solutions of ill-Posed    Problems, W. H. Winston, Washington D.C.-   [24] Evgeniou, T., Pontil, and M. Poggio, T. [2000] “Statistical    Learning Theory: A Primer,” International Journal of Computer    Vision, Vol. 38(1), pp. 9-13.-   [25] T. Evgeniou, M. Pontil, and T. Poggio [2000] “Regularization    Networks and Support Vector Machines,” in Advances in Large Margin    Classifiers, MIT Press.-   [26] Poggio, T., and Smale S., [2003] “The Mathematics of Learning:    Dealing with Data,” To appear in Notices of the AMS, May 2003.-   [27] Suykens, J. A. K. and Vandewalle, J. [1999] “Least-Squares    Support Vector Machine Classifiers,” Neural Processing letters, Vol.    9(3), pp. 293-300, Vol. 14, pp. 71-84.-   [28] Suykens, J. A. K., van Gestel, T. de Brabanter, J. De Moor, M.,    and Vandewalle, J. [2003] Least Squares Support Vector Machines,    World Scientific Pub Co, Singapore.-   [29] Svante Wold, Michael Sjöström, and Lennart Eriksson [2001]    “PLS-Regression: a Basic Tool of Chemometrics,” Chemometrics and    Intelligent Laboratory Systems, 58:109-130.-   [30] Møller, M. F., [1993] “A Scaled Conjugate Gradient Algorithm    for Fast Supervised Learning,” Neural Networks, Vol. 6, pp. 525-534.-   [31] H. Ritter, T. Kohonen, “Self-Organizing Semantic Maps,”    Biological Cybernetics, vol. 61, pp. 241-254, 1989.-   [32] T. Kohonen, Self Organization and Associative Memory, 2nd ed.,    Springer-Verlag, 1988.-   [33] T. Kohonen, “The Self-Organizing Map,” Neurocomputing, 21(1):    1-6, November 1998.-   [34] T. Kohonen, “Thing You Haven't Heard about the Self-Organizing    Map,” IEEE International Conference on Neural Network, vol. 3, pp.    1147-1156, 1993.-   [35] T. Kohonen, “Generalization of the Self-Organizing Map,”    International Joint Conference on Neural Networks, vol. 1, pp.    21-26, 1993.-   [36] G. Deboeck and T. Kohonen, Visual Explorations in Finance with    Self-Organizing Maps, Springer, 2000.-   [37] H. C. Card, G. K. Rosendakl, D. K. Mcneill, and R. D. Mcleod,    “Competitive Learning Algorithms and Neurocomputer Architecture,”    IEEE Transactions on Computers, vol. 47, no. 8, pp. 847-858, August    1998.-   [38] J. S. Kirk, and J. M. Zurada, “Motivation for    Genetically-Trained Topography-Preserving Map,” International Joint    Conference on Neural Networks 2002, vol. 1, pp. 394-399, 2002.-   [39] J. Vesanto, J. Himberg, M. Siponen, and A. Ollisimula,    “Enhancing SOM Based Data Visualization,” Proceedings of the    International Conference on Soft Computing and    Information/Intelligent Systems, pp. 64-67, 1998.-   [40] G. Stroink, W. Moshage, S. Achenbach: “Cardiomagnetism”. In:    Magnetism in Medicine, W. Andrä, H. Nowak, eds. Berlin: Wiley VCH;    1998; 136-189.-   [41] M. Mäkijärvi, J. Montonen, J. Nenonen: “Clinical application of    magnetocardiographic mapping” in: Cardiac mapping, M. Shenasa, M.    Borgreffe, G. Breithardt, Eds. Mount Kisco, N.Y.: Futura Publishing    Co, 2003.-   [42] M. Mäkijärvi, K. Brockmeier, U. Leder, et al.: “New trends in    clinical magnetocardiography”. In Biomag96: Proc. of the 10th    Internat. Conf. on Biomagnetism, Aine C., et al., eds., New York:    Springer, 2000; 410-417.

We claim:
 1. A method for automating the identification of meaningfulfeatures and the formulation of expert rules for classifyingmagnetocardiography data, comprising: applying a direct kernel transformto sensed data acquired from sensors sensing magnetic fields generatedby a patient's heart activity, resulting in transformed data; andidentifying said meaningful features and formulating said expert rulesfrom said transformed data, using machine learning.
 2. The method ofclaim 1, said kernel transform satisfying Mercer conditions.
 3. Themethod of claim 1, said kernel transform comprising a radial basisfunction.
 4. The method of claim 1, said applying a kernel transformcomprising: assigning said transformed data to a first hidden layer of aneural network; applying training data descriptors as weights of saidfirst hidden layer of said neural network; and calculating weights of asecond hidden layer of said neural network numerically.
 5. The method ofclaim 1, further comprising: classifying said transformed data usingdirect kernel partial least square (DK-PLS) machine learning.
 6. Themethod of claim 1, further comprising transforming said sensed data intowavelet domain data prior to applying said kernel transform by: applyinga Daubechies wavelet transform to said sensed data.
 7. The method ofclaim 6, further comprising: selecting features from said wavelet domaindata which improve said classification of magnetocardiography data. 8.The method of claim 1, further comprising: normalizing said sensed data.9. The method of claim 8, said normalizing said sensed data comprising:Mahalanobis scaling said sensed data.
 10. The method of claim 1, furthercomprising: centering a kernel of said kernel transform.
 11. Anapparatus for automating the identification of meaningful features andthe formulation of expert rules for classifying magnetocardiographydata, comprising computerized storage, processing and programming for:applying a direct kernel transform to sensed data acquired from sensorssensing magnetic fields generated by a patient's heart activity,resulting in transformed data; and identifying said meaningful featuresand formulating said expert rules from said transformed data, usingmachine learning.
 12. The apparatus of claim 11, wherein kerneltransform satisfies Mercer conditions.
 13. The apparatus of claim 11,said kernel transform comprising a radial basis function.
 14. Theapparatus of claim 11, said computerized storage, processing andprogramming for applying a kernel transform further comprisingcomputerized storage, processing and programming for: assigning saidtransformed data to a first hidden layer of a neural network; applyingtraining data descriptors as weights of said first hidden layer of saidneural network; and calculating weights of a second hidden layer of saidneural network numerically.
 15. The apparatus of claim 11, furthercomprising computerized storage, processing and programming for:classifying said transformed data using direct kernel partial leastsquare (DK-PLS) machine learning.
 16. The apparatus of claim 11, furthercomprising using said computerized storage, processing and programmingfor transforming said sensed data into wavelet domain data prior toapplying said kernel transform by: applying a Daubechies wavelettransform to said sensed data.
 17. The apparatus of claim 16, furthercomprising using said computerized storage, processing and programmingfor: selecting features from said wavelet domain data which improve saidclassification of magnetocardiography data.
 18. The apparatus of claim11, further comprising computerized storage, processing and programmingfor: normalizing said sensed data.
 19. The apparatus of claim 18, saidcomputerized storage, processing and programming for normalizing saidsensed data comprising computerized storage, processing and programmingfor: Mahalanobis scaling said sensed data.
 20. The apparatus of claim11, further comprising computerized storage, processing and programmingfor: centering a kernel of said kernel transform.